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Introduction 


»  Why  Is  This  Important? 

»  Integral  part  of  an  overall  Economic  Analysis  (EA), 
Return  on  Investment  (ROI) 

»  Program  Justification  to  External  Entities  (Investment 
Review  Boards  and  Other-than-Defense  Agencies) 

»  New  emphasis  on  Program  Value  -  IT  Investment 
Management 

»  Decision  Analysis  -  Selection  of  Preferred  Alternative 
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Introduction 


»  Why  Is  This  Important?  (Cont.) 

»  Internal  Competition  for  Resources  (POM  and  FYDP, 
POPS) 

»  GAO  Audits  and  Tracked  Recommendations  (IG 
reporting  to  Congress) 

»  Acquisition  Requirements  (JCIDS,  Others) 
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Cost  Analysis  and  Benefits  Analysis 
Share  Some  Technical  Points 


»  Cost  Analysis 


» 


» 


» 


» 


» 


» 


Analogy 
Parametric 
Engineering  Buiid-Up 

Expert  Opinion 
Extrapolation  from  Actual 
Cost  Element  Structure  (CES) 


»  Benefits  Analysis 

»  Analogy 
»  Parametric 
»  Engineering  Build-Up 

»  Expert  Opinion 
»  Extrapolation  from  Actual 

»  Benefits  Elements  Matrix 
(BEM) 


Both  analyses  sensitive  to  up-front  important  analytical 
decisions  about  ailocations  and  application  of  economic 
concepts  (NPV,  ROi,  BE) 
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However,  there  are  important 
differences... 


Cost  Analysis 


»  Well  recognized,  published 
sources  of  cost  data  and 
factors 

»  Service  Cost  Agency 
Review  Process 

»  Risk  is  better  understood 
(CSPT) 

»  Common  Accepted  Terms- 
of-Reference  Across  Field 


»  Less  Reliance  on  Expert 
Opinion 

»  Certification  for  Analysts 
»  Cost  Review  Boards 
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However,  there  are  important 

differences..  .  (cont.) 


Benefits  Analysis 


» 


»  Analogies  for  new  programs 
do  not  exist  and  little 
published  benefits  data 

Parameters  are  often 
unknown 

Te  rm  s-of- Refe  re  n  ce 

Heavy  use  of  Expert 
Opinion 


» 


» 


»  Risk  not  well  defined 

»  No  Benefits  Review 
Board 

»  Quality  of  Benefits 
Analysis  dependent  upoi 
client  environment 

»  Less  scrutiny 
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Agenda 


» Introduction 

»  Similarities/Differences  in  Benefits 
Analysis  Approach  vs.  Cost  Analysis 


» 


Benefits  Analysis  Process 


»  Capabilities-Based  Process 
»  References 
»  Acronyms 
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Relative  Differences  Between  Cost 
vs.  Benefits  Analysis  Processes 


»  Risk  Mitigated 

»  Forecast  Cost  from  Solid  Analytical  Base 


Cost  Analysis 


Prospective  Life 
Cycle  Cost 
Estimate 


Cost  Analysis  Training 


Cost  Estimating  Guides 


Cost  Estimating  Relationships  (CER) 


Mostly  Retrospective  Analysis  Elements  Applied  to  Future  State 
^  C3.sk^  @  2011  -  Cask  Technologies  |  Strategy.  Solutions.  Success. 


Relative  Differences  Between  Cost 
vs.  Benefits  Analysis  Processes 


Benefits  Analysis 


» Induced  Risk 

»  Forecast  Benefits  from  Less  Solid  Analytical 

Benefit  Estimating  Relationship^ 
not  clearly  defined 


Intensive  Research  for  Benefits  as 
Related  to  Future  Capabilities 


Few  MAIS  Analogies 


Retrospective 
Benefits  Analysis^ 
Elements 


£CaskL 


Mostly  Prospective  Analysis  Elements  of  Future  State  based  on 
“What  is  to  be,’”  and  Extensive  Use  of  Data  Mining  and  Bl  Tools 
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What  is  an  Economic  Benefits 
Analysis? 


Economic  Analysis  is  required  for  MAIS  programs: 

»  Milestone  A  (may  be  combined  with  Analysis  of  Aiternatives) 
»  Milestone  B  (or  equivalent) 

»  Full  Deployment  Review  (or  f>fiiiivaient) 


“The  purpose  of  the  Economic  Anaiysis  is  to  determine  the  best 
AIS  program  acquisition  alternative,  by  assessing  the  net  costs 
and  benefits  of  the  proposed  AIS  program  relative  to  the  status 


*DoDI  5000.2  Enci  (4),  Tabie  2-1 
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Economic  Analysis  Process 


Data  Collection 


Outputs 


Cost  Analysis 
Requirements 
Description 
(CARD) 


Economic 
Analysis 
Development 
Plan(EADP) 

Establish  Ground 
Rules  &  Assumptions 

Establish  Economic 
Analysis  Approach 


Develop  Status  Quo 
LCCE 


Develop  Preferred 
Alternative  LCCE 


Develop 
Performance 
Improvement 
Metrics  Tabie 


Conduct  Benefits 
Analysis 


PA  LCCE 


Return  on 
Investment  (ROI) 


Net  Present  Value 
(NPV) 


Payback 


Economic  Analysis 
Document 

] 

Benefits  Analysis 
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Cost  Savings/Avoidance 


Benefits  are  Classified  According  to  ^ 
a  4-Tier  Approach* 


» 


» 


Tier  0  consists  of  monetary  cost  savings  determined  by 
subtracting  the  costs  associated  with  the  PA  from  the  current 
cost  to  maintain  the  SQ 

Tier  1  includes  mission  critical  operational  efficiencies  in 
terms  of  cost  avoidances 


» 


» 


Tier  2  includes  achievable  operational  efficiencies 
subordinate  to  the  efficiencies  received  in  Tier  1 ,  but  which 
still  provide  cost  avoidances  to  the  DoD 

Tier  3  consists  of  qualitative  benefits  associated  with  the 
implementation  of  the  Preferred  Alternative 


*Based  on  Guidance  from  OSD  CAPE:  Methodology  captures 
inherent  subjectivity  and  relative  risk.  Cost  savings  increase 
incrementally  and  cumulatively  from  “initial  outlay”  benefits  to  the 
inclusion  of  “must  have”  benefits  to  the  inclusion  of  “nice  to  have” 
benefits. 


CCask, 
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Benefits  Analysis  Methodology  - 
Four  Steps: 


»  Identify  the  benefits  of  the  Preferred  Alternative  (PA)  over  the  Status 
Quo  (SQ) 

»  Develop  the  benefits  model  by  allocating  benefits  to  the  four  tiers  of 
the  framework 

»  Estimate  the  monetary  value  of  the  quantitative  benefits  by 

combining  the  LCCE,  cost  savings,  and  operational  efficiencies  -  this 
will  yield  an  ROI,  NPV,  and  Payback  Period  calculations 

»  Perform  uncertainty  and  sensitivity  analysis  around  the  benefits 
estimate 


Produce  Develop  Estimate  Evaluate 

LCCE  Benefits  Benefits  Risks 

Framework 


£Casl^ 
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Agenda 


» Introduction 

»  Similarities/Differences  in  Benefits 
Analysis  Approach  vs.  Cost  Analysis 

» 

» 

» 

»  Acronyms 


Benefits  Analysis  Process 
Capabilities-Based  Process 
References 
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Capabilities-Based  Benefits 
Analysis  Process 


»  Investment  Provides  Capabilities  which  Generate  Benefits 

»  Return  on  Investment  (ROI)  is  negative  in  the  beginning  due  to 
massive  initial  investment  outlay  for  development  and 
implementation 

»  ROI  improves  as  the  system  increasingly  provides  capabilities  and 
generates  monetary  benefits  as  well  as  intangible  benefits 

»  End  Game:  More  Benefits  than  Cost 
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Capabilities-Based  Benefits: 
System  Investment 


»  Three  Major  DoD  Decision  Support  Systems 

»  Capabilities  Development  through  the  Joint  Capabilities 
Integration  and  Development  System  (JCIDS) 

»  Acquisition  Management  System  (AMS) 

»  Planning,  Programming,  Budgeting  and  Execution 
System  (PPBE) 

»  Decision  to  invest  in  a  new  MAIS  is  a  result  of  JCIDS 
DOTMLPF*  analyses  that  identified  Capability  Gaps,  and 
recommended  investment  in  a  Materiel  Solution  to  achieve 
desired  Operational  Efficiencies 


*Doctrine,  Organization,  Training,  Materiel,  Leadership  &  Education,  Personnel,  Facilities 


£Casl^ 
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Capabilities-Based  Benefits: 

System  Investment  (cont.) 


»  JCIDS  and  AMS  require  documents  that  articulate  the 
capabilities  that  will  be  created  through  investment  in  the 
MAIS 

»  Economic  Analysis  of  both  Costs  and  Benefits  provide 
decision  makers  with  the  information  needed  to  make 
resource  allocation  decisions  among  the  projects  in  the 
portfolio,  given  the  proposed  capability  gains  (decision  is 
not  always  economics  based) 
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Capabilities-Based  Benefits: 
Operational  Capabilities 


»  Source  document  for  Capabilities-Based  Benefits  Analysis 
is  determined  by  MAIS  Program’s  position  on  the 
Acquisition  lifecycle 


CCask, 
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Capabilities-Based  Benefits: 
Operational  Efficiencies 


»  To  the  extent  possible: 

»  Baseline  current  As-ls  state  in  relation  to  the  capabilities 
of  the  new  system 

»  Determine  which  Capability  Sets  hold  the  most  promise 
for  benefits  and  bin  according  to  a  must-have/nice-to- 
have  thought  process,  and  determine  how  they  will  be 
analyzed 

»  Utilize  Data  Analysis  Plan,  Data  Mining  Tools,  Business 
Intelligence  Tools 
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Example:  Inventory  Reduction 
Benefit 


Increase  Asset  Visibility  and  Reduce 
Logistics  Footprint 


Improved  Inventory  Ordering  Mgt  and 
Velocity 


Inventory  Reduction  and  Inventory 
Carrying  Cost  Reduction 


Performance  Improvement  Metrics  Table 


New  System  Capabilities 
vs.  As-ls  System 


Performance 

Improvements 


Description  of  Performance 
Improvements 

Capability  Affected 

Current  System  Performance 
(As-is) 

New  System 
Performance 
(To-Be) 

Performance 

Improvement 

Inventory  Reduction 

inventory  carrying 
requirements,  order  timing, 
and  costs 

■Request  Management 

■Supply 

■Maintenance 

Replenishment  action  initiated 
whenever  on-hand  quantity  plus 
the  dues,  less  back  orders,  is 
equal  to  or  less  than  the  reorder 
point.  Inflates  Inventory 

System  will  use  advanced 
supply  chain  planning  engine 
at  retail  level  and  min/max 
planning  tool 

Potential  decrease  in 
Inventory  and 

Inventory  Carrying 

Cost 
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Capabilities-Based  Benefits: 
Operational  Efficiencies 


»  Translate  selected  new  capabilities  into  Operational 
Efficiencies 

»  Metrics  for  quantitative  cost  savings  and  cost 
avoidances  in  terms  of  reduction  in  FTEs,  reduced 
order-to-ship  time,  order  cycle  time,  increased  customer 
satisfaction,  etc. 

»  Document  non-monetary  benefits  through  research  and 
facilitated  expert  panels 

»  Identify  the  top  ten  benefits  for  potential  to  generate: 

»  Cost  Savings 

»  Cost  Avoidances 

»  Intangible  or  Non-monetary  benefits 


£Casl^ 
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Example:  Inventory  Reduction 

Benefit  (Cont.) 


Perforrmnce  lirfDroN/eiTient  Metrics  Table 


Description  of  Performance 
Improvements 

Capability  Affected 

Current  System  Performance 
(As-is) 

New  System 
Performance 
(To-Be) 

Performance 

Improvement 

Inventory  Reduction 

Inventory  carrying 
requirements,  order  riming,  and 
costs 

■Request  Management 

■Supply 

■Maintenance 

Replenishment  action  initiated 
whenever  on-hand  quantity  plus 
the  dues,  less  back  orders,  is  equal 
to  or  less  than  the  reorder  point. 
Inflates  Inventory 

System  will  use  advanced 
supply  chain  planning  engine 
at  retail  level  and  min/hiax 
planning  tool 

One  time  drawdown 
between  15-25%, 
Inventory  Carrying 

Cost  reduction  of  25% 
annually 
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Capabilities-Based  Benefits: 
Monetized  Cost  Avoidances 


»  Express  Operational  Efficiencies  in  terms  of  money 

»  Number  of  units  reduced  by  cost  of  each  unit 

»  Expected  percentage  reductions  over  time  priced  by  the 
proportion 

»  Standard  factors  applied  to  baseline  quantities 

»  Any  other  reasonable  monetary  conversion  of  an 
expected  efficiency  gain  generated  by  the  investment  in 
a  capability 

»  Document  results  and  methodologies  used 

»  Relate  results  to  capability 

»  Be  cognizant  of  the  difference  between  Cost  Savings  and 
Cost  Avoidance  ____ 


£Casl^ 
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Capabilities-Based  Benefits: 
Monetized  Cost  Avoidances 


Performance  Improvement  Metrics  Table 


Cost  Savings 

Cost  Avoidance 

Direct  cost  saving  result  now  as  a  result  of  the  action 

j  Cost  that  will  be  incurred  in  the  future  if  the  action  is  nottaken 

Reduced  Appropriation  Baseline 

Inventory  Carrying  Costs 

Reduced  Budget  Spend  Plan 

Maintenance  Actions 

Volume  Reduction 

Personnel  Expenses 

Description  of 
Performance 
Improvements 

Capability  Affected 

Current  System  Performance 
(As -is) 

New  System 
Performance 
(To-Be) 

Performance 

Improvement 

Yearly  Benefit 

Inventory  Reduction 

Inventory  carrying 
requirements j  order  timing, 
and  costs 

■Request  Management 

■Supply 

■Maintenance 

Replenishment  action  initiated 
whenever  on-hand  quantity  plus 
the  dues,  less  back  orders,  is 
equal  to  or  less  than  the  reorder 
point.  Inflates  Inventory 

System  will  use  advanced 
supply  chain  planning 
engine  at  retail  level  and 
min/max  planning  tool 

i 

One  time  drawdown 
between  15-25%,  i 

1 

Inventory  Carrying  \ 
Cost  reduction  of  \ 
20%  annually 

\ 

One  time  $500M 
drawdown  of 
inventory  spread 
over  3  years 

Annual  Inventory 
Carrying  Cost 
Avoidance  of  $100M 
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Capabilities-Based  Benefits: 
Benefit  Summary/ROI 


»  Gather  all  benefits  expressed  in  terms  of  money 

»  Determine  the  time-phasing  as  to  when  each 
benefit  will  be  actualized  by  year 

»  Does  the  benefit  begin  at  IOC  or  FOC? 

»  How  long  do  you  expect  to  recognize  the  benefit? 
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Capabilities-Based  Benefits: 
Benefit  Summary/ROI 


»  Are  there  any  allocation  concepts  that  can  be 
applied  such  as  those  found  in  Cost  Accounting? 
Other  recognized  sources? 

»  Apply  financial  tools  to  calculate  and  report  Net 
Present  Value  (NPV)  and  Return  On  Investment 
(ROI)  using  the  Benefits  Data,  and  Life  Cycle  Cost 
Estimate  (LCCE) 

»  Prepare  for  Post-Investment  Analysis  (PIA) 

»  How  well  did  we  do  against  expected  benefit 
generation  and  expected  ROI? 
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Capabilities-Based  Benefits:  Keys 
to  Success 

»  Program  Office  Support 

»  Benefits  IPT  (separate  from  the  Cost  IPT) 

»  Start  with  the  Program  Capabilities/Requirements 
Documents 

»  Document  traceability 

»  Gain  Buy-In  from  Functional  Advocate,  Legacy  System 
Owners,  and  Stakeholders 

»  Identify  monetized  benefits  that  can  offset  future  budget 
request/POMs 

»  Use  actual  data  (if  possible) 

»  Create  a  benefits  capture  plan  up  front 

£Casl^ 
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»  DoD  Automated  Information  Systems  Economic  Analysis  Guide  -  1995: 
httD://www.ncca. navv.mil/resources/DoD  AIS  EA  Guide  1995  DRAFT  R 

CW.Ddf 

»  Analysis  of  Alternatives  (AoA)  Handbook:  Practical  Guide  to  Analyses  of 

Alternatives,  July  2008,  Office  of  Aerospace  Studies,  Air  Force  Materiel 
Command  (AFMC)  OAS/A9,1655  1st  Street  SE,  Kirtland  AFB,  NM  87117- 
5522;  http:/ /WWW,  oas.  kirtland.  af.  mil/ 

»  Department  of  the  Army  -  Economic  Analysis  Manual: 

WWW,  asafm.  army,  mil/pubs/cdfs/manual/economic.  pdf 

»  Benchmarkina  Cost  Savinas  &  Cost  Avoidance  -  NASPO 

BENCHMARKING  WORKGROUP  Research  Brief  September  2007; 
http:/ /WWW,  naspo.  ora/old  site/whitepapers/documents/BenchmarkingCostS 

a  vingsandCostA  voidance.pdf 

»  There  are  many  more  resources,  but  these  are  a  good  start. 
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Acronyms 


Acronym 

Definition 

Acronym 

Definition 

AIS 

Automated  Information 

System 

CPD 

Capabilities  Production  Document 

AMS 

Acquisition  Management 
System 

CSPT 

Cost,  Schedule,  Performance,  Technology 

BE 

Break  Even 

DoDI 

Department  of  Defense  Instruction 

BEM 

Benefits  Element  Matrix 

DOTMLPF 

Doctrine,  Organization,  Training,  Materiel, 
Leadership,  Personnel,  Facilities 

CAPE 

Cost  Analysis  and  Program 
Evaluation 

EA 

Economic  Analysis 

CARD 

Cost  Analysis  Requirements 
Description 

EADP 

Economic  Analysis  Development  Plan 

CDD 

Capabilities  Development 
Document 

FOC 

Full  Operational  Capability 

CER 

Cost  Estimating  Relationship 

FTE 

Full  Time  Equivalent 

CES 

Cost  Element  Structure 

FYDP 

Future  Years  Defense  Plan 
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Acronyms 


Acronym 

Definition 

Acronym 

Definition 

GAO 

Government  Accountability 
Office 

NPV 

Net  Present  Value 

ICD 

Initial  Capabilities  Document 

OSD 

Office  of  the  Secretary  of  Defense 

IG 

Inspector  General 

PA 

Preferred  Alternative 

IOC 

Initial  Operating  Capability 

PIA 

Post  Investment  Analysis 

IPT 

Integrated  Product  Team 

PIMT 

Performance  Improvement  Metrics  Table 

IT 

Information  Technology 

POM 

Program  Objective  Memorandum 

JCIDS 

Joint  Capabilities  Integration 
and  Development  System 

Pops 

Probability  of  Program  Success 

LCCE 

Life  Cycle  Cost  Estimate 

PPBES 

Programming,  Planning,  Budgeting,  and 
Execution  System 

MAIS 

Major  Automated  Information 
System 

ROI 

Return  on  Investment 

SO 

Status  Quo 
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